test: setup go-vcr to start recording real providers interactions#6
Conversation
andreynering
force-pushed
the
charm-605-write-integration-tests-that-uses-the-real-provider-apis
branch
2 times, most recently
from
4048173 to
d986aa6
Compare
andreynering
requested review from
a team,
aymanbagabas,
Copilot and
kujtimiihoxha
and removed request for
a team
There was a problem hiding this comment.
Pull Request Overview
This PR sets up go-vcr for recording real provider interactions in testing, enabling deterministic test replays without making live API calls. The PR introduces a comprehensive test suite for multiple AI providers (OpenAI and Anthropic) with test scenarios covering simple generation, tool usage, streaming, and streaming with tools.
- Added go-vcr dependency and HTTP recording infrastructure
- Created provider tests with support for OpenAI GPT-4o/4o-mini and Anthropic Claude Sonnet models
- Generated test recordings for all supported providers across different test scenarios
Reviewed Changes
Copilot reviewed 18 out of 19 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| go.mod | Adds dependencies for go-vcr and godotenv for test recording and environment management |
| .gitattributes | Excludes generated YAML test data from diffs and marks as linguist-generated |
| providertests/.env.sample | Provides template for required API keys |
| providertests/builders_test.go | Defines language model builders for different AI providers |
| providertests/provider_test.go | Implements comprehensive test scenarios for AI provider functionality |
| providertests/recorder_test.go | Sets up VCR recording infrastructure with custom matchers and header filtering |
| providertests/testdata/**/*.yaml | Generated cassette files containing recorded HTTP interactions for test replay |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
kujtimiihoxha
left a comment
kujtimiihoxha
left a comment
There was a problem hiding this comment.
Love the way you set it up so it tests the same things for different providers.
This makes sense because we want it to work the same for all language models.
Long term we want to add more tests to make sure that we are parsing the messages correctly and we want to test more multi step scenarios.
andreynering
force-pushed
the
charm-605-write-integration-tests-that-uses-the-real-provider-apis
branch
2 times, most recently
from
0b5f18a to
6a8dbdf
Compare
andreynering
deleted the
charm-605-write-integration-tests-that-uses-the-real-provider-apis
branch
Not a big deal, but in the progress I opened the following PR to
go-vcrto allow us to have multi-line formatted strings in the YAML files. This is useful improve legibility, in particular when streaming.WithEncodeOptionto tweak yaml marshalling dnaeon/go-vcr#129